One of the first questions that arise where deciding to set up a tunnel between two tunnel end-points is that of knowing which should be the transport protocol of this tunnel.
In the prior art, it is chiefly the layer 3 IP (Internet protocol) or the layer 4 TCP/UDP (transmission control protocol/user datagram protocol) that is used. Since IP-based tunnelling technologies cannot take account of the network address translation (NAT) mechanism and since they are not entirely compatible with the typical tunnelling configuration of FIG. 1, in the rest of the description here below, we consider (by way of examples only) solutions based on layer 4 (transport layer) i.e. on the TCP or the UDP.
The TCP protocol is a connection-oriented reliable protocol giving the sender the guarantee that his or her data will be effectively received (management of acknowledgement) and that all the frames are received in a given order. The TCP applies an efficient congestion control mechanism.
The UDP protocol is a far simpler and faster protocol that does not take account of the order of the frames and does not manage acknowledgement.
In this particular case (referred to solely by way of an example), the above-mentioned question becomes one of knowing whether the TCP or UDP protocol should be used as a transport protocol for the tunnel?
The problem is that the protocol corresponding to the data used in the passenger protocol may interfere with the mechanisms implemented in the transport protocol in the tunnel. For example, if we look at the TCP as a transport protocol and the TCP as a protocol corresponding to the payload data of the passenger protocol (the combination known as TCP over TCP), we are confronted with destructive interactions between the two TCP congestion control mechanisms. For further details, reference may be made especially to “Understanding TCP over TCP: effects of TCP tunnelling on end-to-end throughput and latency (O Honda, H Ohsaki, M. Imase, M. Ishizuka, J. Murayama). (Proceedings of the SPIE, volume 6011, pp 138-146 (October 2005)”.
A first response may be to say that the TCP over TCP combination is not a good solution. However, even if, in certain conditions, it is well known that this type of tunnelling downgrades end-to-end performance, under other conditions the same combination improves end-to-end performance (see for example the above-mentioned document as well as the following document: “Avoiding congestion collapse on the Internet using TCP tunnels (B. P. Lee, R. K. Balan, L. Jacob, W. K. G Seah, A. L Ananda) (Computer Networks 39 (2002) pages 207-219, December 2002)”. The same problem arises with the “UDP over UDP” combination, i.e. when we look at the UDP protocol as the transport protocol and the UDP protocol as the passenger protocol.
There is therefore no absolute response to the above-mentioned question (namely which is the transport protocol to be used in the tunnel) because this depends essentially on three factors:                the type of data to be transmitted through the tunnel (protocol corresponding to the payload data of the passenger protocol, type of application (transfer of files, audio and/or video streaming etc);        the quality of the network (in terms of frame loss or corruption, congestion etc) between the two tunnel end-points; and        the user's and/or administrator's preferences (in terms of bandwidth, reliability, jitter etc).        
At present, when it is decided to set up a tunnel between two tunnel end-points, it is imperatively necessary to make a predetermined choice for the transport protocol (i.e. a predetermined choice of channel in the tunnel, should each channel use a distinct transport protocol) although this choice is not optimal in all situations.
There is a known technique, described in the US patent document No. 6614800 that uses two virtual private networks (VPNs), i.e. two tunnels: the first tunnel (between two IP addresses) for control data, the second panel (between two other IP addresses) for the payload data. This technique enables the choice of a first transport protocol for the control data and a second transport protocol for the payload data, the two types of data going through two distinct tunnels. The choice of the transport protocol may therefore be optimized on each of the two channels.
This technique however has two major drawbacks: it calls for two tunnels (two pairs of IP addresses) and each type of data (control data and payload data) always uses the same transport protocol. For a type of data considered, the choice is therefore not optimum in every situation (we shall return to the discussion here below).